… step 1/3)

First slice of the governance proposal creation feature. Pure backend,
no wiring yet — appFactory + mailer template land in the next commit so
CI can exercise these modules in isolation first.

Summary:

- `lib/proposalHash.js` — JS port of Syscoin Core's
  `CGovernanceObject::GetHash()`. We need this because Core only
  exposes the hash via `gobject_prepare`, which ALWAYS burns 150 SYS
  from the node wallet. Computing it locally lets the dApp hand the
  OP_RETURN bytes to the user's own wallet (Pali or Syscoin-Qt) and
  keep the backend wallet-less. Verified against a frozen golden
  vector plus sensitivity tests.

- `lib/proposalValidate.js` — canonical JSON serializer + structural
  validator. Emits exactly the on-chain fields (type, name,
  start_epoch, end_epoch, payment_address, payment_amount, url) with
  a fixed key order and non-scientific decimal amounts so the hash
  is byte-deterministic. Stricter than Core on URL scheme
  (requires http(s)://) as a social-engineering guardrail.

- `db/migrations/001_init.sql` (in-place) — `proposal_drafts` and
  `proposal_submissions` tables. Submissions snapshot the canonical
  hashing fields (parent_hash, revision, time_unix, data_hex,
  proposal_hash) as immutable; the collateral OP_RETURN commits to
  these, so freezing them at create() time prevents drift.

- `lib/proposalDrafts.js` + tests — plaintext per-user drafts,
  BigInt-safe amounts (.safeIntegers(true)).

- `lib/proposalSubmissions.js` + tests — state machine
  (prepared → awaiting_collateral → submitted | failed) enforced
  inside the repo with stable `.code` errors.

- `lib/proposalDispatcher.js` + tests — background advance loop:
  polls `getRawTransaction`, bumps conf counts, fires
  `gObject_submit` once >= GOVERNANCE_FEE_CONFIRMATIONS (6) confs.
  Classifies Core errors as terminal (→ failed) vs transient (retry)
  and times out rows whose collateral is missing > 7 days.

- `routes/govProposals.js` + tests — HTTP surface:
  * drafts CRUD (50-per-user soft cap)
  * POST /prepare — canonicalize, compute hash, optional
    gObject_check pre-flight, persist `prepared` row (idempotent on
    (userId, proposalHash)), consume source draft in an atomic block
  * POST /submissions/:id/attach-collateral — hand-off to dispatcher
  * GET/DELETE for status polling and cleanup

Notes:

- `payment_count` is NOT a consensus field (Core has no bound on it;
  number of payouts is implicit from (end_epoch - start_epoch) /
  superblock_cycle). We keep it as pure display metadata with a soft
  1..60 range check in the route layer — never in canonical JSON.

- We submit after 6 confs rather than relying on Core's
  "postponed on immature collateral" path because it gives a cleaner
  UX: one deterministic state transition instead of a silent
  Core-internal retry schedule.

Tests: 658/658 passing, 5 consecutive clean full-suite runs.
Made-with: Cursor

…bmit)

Two review findings from Codex on PR #8 (backend):

P1 (routes/govProposals.js): the /prepare pre-flight called
rpc.gObjectCheck with only dataHex, but server.js defines the
adapter with the full Core signature
(parentHash, revision, time, dataHex). That shifted args so dataHex
became parentHash and the real payload was undefined; subsequent
invalid-params errors from Core matched the /invalid/ classifier
and masqueraded as 422 core_rejected for otherwise valid
proposals. Fix: always pass the full canonical tuple we just
hashed. Regression test asserts on the exact arg shape.

P2 (lib/proposalDispatcher.js): /already exists/ was in the
TERMINAL_CORE_ERRORS list, so a dispatcher retry after a prior
successful gObject_submit (crash before markSubmitted, overlapping
workers, operator running two dispatchers, etc.) would flip the
row to failed and fire a false failure email, even though the
governance object is live on-chain. Fix: classify duplicate-submit
as a transient error, log it loudly via submit_already_exists, and
let the next tick retry so the existing governance_hash_clash path
in the status handler can promote it to submitted naturally.
Added a test that asserts the row stays awaiting_collateral and
no failure transition fires.

Full BE suite green (29 suites / 674 tests).

Made-with: Cursor

…te-submit resolution)

Two new P1 findings from Codex round 2 on PR #8 — the first-round
fixes opened up secondary issues that only showed up under retry
scenarios. Both are real production hazards, not just reviewer
taste.

P1 #2 (routes/govProposals.js, lib/proposalSubmissions.js):
`proposalHash` bakes in `time` (seconds since epoch) — part of the
CGovernanceObject::GetHash() input tuple Core signs under. The
old idempotency check keyed /prepare on `proposalHash`, so two
retries of the same logical /prepare that straddled a one-second
boundary hashed differently and BOTH landed in the DB as
`prepared`, leaving the user with duplicate pending submissions
for one proposal. Fix: add findPreparedByDataHexForUser() on the
submissions repo (scoped to owner, status=prepared, ordered by
recency) and make /prepare look up by the time-free canonical
payload first. On hit we reuse the frozen {time, proposalHash}
from the existing row, skip the RPC pre-flight (Core already
accepted it and rate-limits repeats), and return 200 idempotent.
Regression test injects a monotonic now() and advances the
clock >1s between two identical POST /prepare calls, asserts
single row, same id, same hash, same time.

P1 #3 (lib/proposalDispatcher.js): the round-1 fix that stopped
flipping "already exists" to `failed` left the row in
`awaiting_collateral` forever if Core kept returning the
duplicate error — no terminal transition, no completion signal
to the user. Core's govobj store is keyed on
CGovernanceObject::GetHash(parent, rev, time, vchData, outpoint,
sig), which computeProposalHash() reproduces exactly at prepare
time (outpoint/sig are empty for top-level user proposals), so
the on-chain hash MUST equal row.proposalHash. Resolve by
calling markSubmitted({ governanceHash: row.proposalHash })
inside the duplicate branch. If the repo rejects with
governance_hash_clash (rare race against a parallel tick or a
coincidentally-identical submission), we log and leave the
state machine to reconcile — still no false-failure email, but
now we have a real terminal path. Two new tests cover both
outcomes.

Full BE suite green (29 suites / 676 tests).

Made-with: Cursor

Address the two P2 findings from Codex's latest pass on PR #8 by
turning the read-before-write idempotency/duplicate checks into
DB-enforced invariants and normalising the remaining SQLite
constraint errors into stable application-level codes.

1. /prepare — payload-keyed idempotency is now atomic (Codex P2)
   - New partial unique index on proposal_submissions:
     (user_id, data_hex) WHERE status = 'prepared'. Two concurrent
     /prepare calls with identical canonical payloads can no longer
     both insert; the second write is rejected at the SQLite layer.
   - Route catches SQLITE_CONSTRAINT_UNIQUE from the create() path
     and rehashes/returns the winner's row as a 200 idempotent
     response, matching the pre-read idempotent branch. Caller can't
     tell whether the pre-read or the constraint fallback served it,
     which is the whole point.

2. attachCollateral — write-level race normalised (Codex P2)
   - Repo wraps the status='awaiting_collateral' UPDATE in a
     try/catch; SQLite's partial unique index on collateral_txid
     translates to the same `txid_already_used` code that the
     pre-read clash throws, so the route layer keeps returning 409
     regardless of which branch fires.

3. Pre-launch migration convention is preserved (Codex P1)
   - Codex flagged the new DDL as "needs a 002_ migration". The
     header in db/migrations/001_init.sql explicitly opts out of
     that rule pre-launch — 001 is the single source of truth until
     v1 ships. The new partial index lives in 001_init.sql in
     keeping with that convention. (Will flip to 002+ files for
     every schema change post-launch.)

Tests
- New: repo test that two prepared rows with same (user_id, data_hex)
  are rejected at the DB layer; different users may each hold a
  prepared row with the same dataHex; the partial predicate releases
  once the first row leaves 'prepared'.
- New: repo test that simulates the attach-collateral race by
  planting a competing row between the pre-read and the UPDATE; the
  raw SQLite constraint error is translated to txid_already_used.
- New: route test that forces the pre-read miss once and verifies
  the /prepare constraint fallback returns 200 idempotent with the
  winner's submission and the DB still holds exactly one row.
- Adjusted existing tests that happened to rely on being able to
  create two prepared rows with the same dataHex (listByStatus,
  proposalDispatcher multi-row test, db.test.js collateral_txid
  uniqueness fixture) to use distinct data_hex values — the intent
  of each test is unchanged.

All 682 backend tests green.

Made-with: Cursor

Adds a minimal GitHub Actions workflow that installs with `npm ci`
and runs `npm test -- --runInBand --ci` on Node 20 and Node 22
(both in LTS). --runInBand keeps the SQLite test suites
deterministic on the hosted runner and avoids the rare cross-test
`:memory:` db leakage we saw locally on multi-worker jobs.

Also adds a cheap schema-sanity job that fails if two migration
filenames share the same numeric prefix — the runner in `lib/db.js`
sorts lexicographically, so colliding prefixes make apply order
undefined. Exactly the kind of silent footgun CI should catch.

concurrency: ci-<workflow>-<ref> with cancel-in-progress so a
force-push supersedes the previous run rather than queueing.

Made-with: Cursor

…odex R5 P1)

Two related prepare-route fixes:

1. `rpc` is advertised as optional and `appFactory.js` explicitly passes
   `null` when no Core connection is wired. Destructured defaults only
   fire for `undefined`, so an explicit null flowed through and the bare
   `typeof rpc.gObjectCheck` dereference threw TypeError inside an async
   handler, surfacing as an unhandled rejection instead of a clean
   "skip preflight". Guard with `rpc &&` before dereferencing.

2. The idempotent-replay branch short-circuited before re-running the
   Core preflight. If the original /prepare landed during a transient
   RPC outage (the code soft-allows network errors), every retry would
   replay the cached row and never revalidate once Core recovered. A
   Core-invalid proposal could then proceed to burn 150 SYS before
   failing in the dispatcher. Moved preflight BEFORE the idempotency
   short-circuit so every /prepare response is backed by a fresh Core
   ack (or an explicit soft-fail we logged). Added regression test.

Made-with: Cursor

…untSats validation (Codex R5 P1/P2)

Backend round 5 follow-ups from the Codex review loop on PR8:

* markSubmitted now does a compare-and-swap on status
  (`WHERE id = ? AND status = 'awaiting_collateral'`). Losing workers
  get back `null` instead of throwing, so two dispatchers that race on
  the same row no longer both emit `onSubmitted` side effects.
* proposalDispatcher handles both paths of the CAS:
  - null return (CAS miss): log `markSubmitted_raced` with
    `code: 'cas_miss'`
  - exception (`status_not_awaiting` / `governance_hash_clash`): log
    once and set a guard so the CAS-miss branch does not re-log.
* /gov/proposals/drafts POST + PATCH validate `paymentAmountSats`
  before forwarding to the repo. Previously malformed values
  (`"12.5"`, `"abc"`, `""`, `"-1"`, `"007"`, `"1e3"`, ...) bubbled out
  of `toBigIntSats()` as a generic 500. Route layer now returns
  the canonical `400 { error: 'validation_failed', issues: [...] }`
  shape with `code: 'amount_sats_invalid'`.

Regression tests added:
* proposalSubmissions.test.js — markSubmitted CAS miss returns null
  without throwing and preserves the winning row's state.
* govProposals.routes.test.js — 400 for malformed paymentAmountSats
  on both POST and PATCH; happy-path digit string still accepted.

All 688 backend tests pass.

Made-with: Cursor

…s": "OK" } (Codex R6 P1)

Two tightly-coupled Codex findings against the proposal preflight,
both surfaced as "otherwise-valid proposals get 422 core_rejected on
a real Core node":

1. Adapter signature: Syscoin Core's `gobject_check` RPC takes
   exactly ONE positional arg, `hex_data` (see
   syscoin/src/rpc/governance.cpp::gobject_check — it constructs
   the govobj with uint256() / 1 / GetAdjustedTime() internally and
   derives nothing from the caller). Our adapter was modelled on
   `gobject_submit` and forwarded (parentHash, revision, time,
   dataHex); Core rejects that with RPC_INVALID_PARAMS ("too many
   positional arguments") and the route's terminal-error classifier
   then flagged that as 422 core_rejected. Adapter + route now pass
   just `canon.dataHex`.

2. Success response shape: Core returns
     { "Object status": "OK" }
   (literal key with a space; governance.cpp line 111). Our check
   looked for `result.Object === 'success'`, which never matches a
   real Core response, so every successful preflight fell through
   to the reject branch. Route now accepts the real shape, with a
   case-insensitive compare on the OK string for forward-compat
   but a strict key match.

Regression tests:
* happy-path preflight mock updated to produce the real shape and
  asserts the adapter sees exactly one arg (the hex_data).
* new: "accepts Core's canonical success shape" — sends the exact
  "Object status: OK" response and expects 201.
* new: "gObjectCheck is called with hex_data only" — pins the
  adapter signature at the route boundary so a future regression
  would fail fast.
* existing "core_rejected when gObjectCheck returns non-success"
  updated to a rejection shape that no longer pretends to set
  `Object: 'failure'` (Core never emits that key).

All 690 backend tests pass.

Made-with: Cursor

… a row (Codex R7 P1/P2)

Two race-safety follow-ups against the proposal submission lifecycle:

* `markFailed` had a read-time status check but the UPDATE only filtered by
  id, so two dispatcher workers processing the same row could have one
  succeed in `markSubmitted` while the other's "terminal reject" path
  (e.g. gObjectSubmit throws a validation-ish error) stomps it back to
  `failed` — corrupting the terminal state and firing onFailed after an
  onSubmitted already went out. Now UPDATE gates on
  `AND status NOT IN ('submitted', 'failed')` and returns null on a CAS
  miss. Dispatcher's `if (failedRow)` guard already skips the hook on
  null, so the winner keeps emit-once semantics.

* `DELETE /submissions/:id` pre-reads the row, confirms it is
  `prepared` or `failed`, then calls `submissions.remove`. That remove
  uses a partial DELETE guarded on the same status set, so a concurrent
  transition (another tab attach-collateral, dispatcher pickup) between
  pre-read and DELETE produces `changes = 0`. The handler used to
  return 204 regardless — a false-success that tells the client the
  submission is gone while it is in fact still live and may run to
  completion on-chain. Route now checks the row count, re-reads on a
  miss, and returns 404 (row deleted by another tab) or 409
  status_not_deletable (raced to a non-deletable state).

Regression tests:
* proposalSubmissions.test.js — CAS-miss on markFailed returns null
  without throwing; winning `submitted` row is preserved.
* govProposals.routes.test.js — DELETE on a row raced to
  `awaiting_collateral` returns 409 and the row is still visible.
* govProposals.routes.test.js — DELETE that finds 0 changes because
  the row was consumed elsewhere returns 404.

All 693 backend tests pass.

Made-with: Cursor

Codex PR8 round 8 fixes.

P1 (routes/govProposals.js): the async /prepare handler had multiple
persistence calls (findPreparedByDataHexForUser,
drafts.getByIdForUser) outside any try/catch. A synchronous
better-sqlite3 throw (SQLITE_BUSY, I/O, corrupt-index) became an
unhandled promise rejection rather than a controlled JSON 500 —
Express 4 does not catch async handler rejections. Wrap both calls
so transient DB faults surface as clean 500s the client can retry.
Regression test forces SQLITE_BUSY on the lookup and asserts a 500
with no unhandled rejection leak.

P2 (lib/appFactory.js): stopProposalDispatcher() only called
clearTimeout on a pending timer. If a tick was already in flight
(past `await dispatcher.tick()`), the callback still re-armed the
loop after resolving, so stop was effectively ignored — leaking the
dispatcher into teardown and shutdown paths. Add a
`dispatcherStopped` flag, check it before AND after the in-flight
tick, and set it in stopProposalDispatcher. Regression test stubs
the dispatcher with a controllable deferred, stops while a tick is
parked on it, then releases it; asserts no further ticks fire.

Made-with: Cursor

…ns (Codex R10 P1/P2)

Two backend fixes from Codex PR8 round 10 review:

P1 (lib/proposalDispatcher.js) — duplicate governance-hash clashes
must transition out of awaiting_collateral. When Core returns
"already exists" and markSubmitted then throws
governance_hash_clash (another row already claimed this
governance_hash via the UNIQUE index), round 9 code only logged
and left the row in awaiting_collateral — so the dispatcher
retried that same row every tick forever, producing no terminal
user-visible outcome even though the on-chain object exists and
the collateral is already spent. Now flip the clashing row to
terminal `failed` with reason `duplicate_governance_hash` via the
same CAS path used elsewhere, and fire onFailed so the user gets
a clean notification. A follow-up tick confirms the terminal row
is no longer picked up.

P2 (lib/proposalDispatcher.js) — tighten TERMINAL_CORE_ERRORS.
The previous list had a blanket /invalid/i pattern that was far
too broad: transient RPC transport / JSON-parser errors ("invalid
JSON-RPC response", "invalid response from server", "invalid
utf-8 sequence", etc.) frequently contain the word "invalid",
and classifying those as permanent rejections flipped rows to
`failed` on temporary outages and fired user-visible failure
emails the user could never fix. Narrow to the exact phrases
Syscoin Core actually emits for governance-object validation
failures (`Governance object is not valid`, `Invalid parent
hash`, `Invalid (object )?signature`, `Invalid object type`,
`Invalid proposal`, `Invalid data hex`, `hash mismatch`) plus the
existing `Object submission rejected` and `rate limit`. Anything
else — bare "invalid" in a transport string — stays classified
as transient and is retried next tick.

Updated an existing test that asserted the hash-clash row stayed
in awaiting_collateral; added a new regression test asserting
the terminal + onFailed + no-retry behavior. Added a separate
regression test for the invalid-transport vs. Core-terminal
distinction. Full BE suite 697/697 passing.

Made-with: Cursor

…ash races

P1 (routes/govProposals.js): narrow gObjectCheck terminal-error
matcher. Previous heuristic included a bare /invalid/ token, which
matched JSON-RPC transport/parser errors that routinely contain the
word "invalid" ("Invalid URL", "invalid response from server",
"invalid JSON-RPC response", "invalid utf-8 sequence in headers").
Those are transient outages, not Core validation rejects, but the
old matcher misreported them as permanent 422 core_rejected and
blocked legitimate /prepare calls until the node/network recovered.

Replaced with an explicit allowlist of phrases Syscoin Core actually
emits from CGovernanceObject::IsValidLocally() and gobject_check's
reject branches (name exceeds, data exceeds, url invalid,
payment_address invalid, start_epoch, type field not found, ...)
plus the gobject-submit wrapper rejects (Governance object is not
valid, Object submission rejected, Invalid parent hash, signature/
object-type/data-hex rejects, collateral rejected). Everything
else falls through to soft-allow. Deliberately does NOT use
parseCoreRejectMessage for classification because its catch-all
arm raises a generic core_rejected issue for ANY non-empty string,
which would false-positive every transport error. Added 11
regression tests: 5 transport-error strings that must NOT terminate
and 6 genuine Core reject phrases that still must 422.

P2 (lib/proposalSubmissions.js): normalize raw SQLITE_CONSTRAINT_
UNIQUE on governance_hash to governance_hash_clash. Under
concurrent dispatcher workers, the read-above (byGovHashStmt
pre-check) + write-here (UPDATE) pair is not atomic — two workers
can each see clash=null, then the second UPDATE hits the partial
UNIQUE index on `governance_hash` and rejects with SQLITE_CONSTRAINT_
UNIQUE. Pre-fix the raw error bubbled up unwrapped; the dispatcher's
duplicate-recovery branch only treats code === 'governance_hash_
clash' as a terminal clash, so the raw error left clashed=false,
the row stayed awaiting_collateral, and every subsequent tick
re-hit Core's "already exists" and looped forever with no
terminal user-visible outcome. Wrap the UPDATE in try/catch and
rethrow as governance_hash_clash (same normalization pattern as
attachCollateral's collateral-txid unique clash). Regression test
exercises the code path by patching db.prepare to force the
pre-check to miss so the UPDATE is the boundary that raises.

Made-with: Cursor

…-read

P1 (lib/proposalDispatcher.js): replace bare /rate limit/i in
TERMINAL_CORE_ERRORS with the exact Syscoin Core phrase
"Object creation rate limit exceeded" (thrown from gobject(submit)
at syscoin/src/rpc/governance.cpp:204). The old pattern matched
transient proxy/provider 429 responses whose bodies routinely
contain "rate limit"/"rate-limited" — those should be retried,
not terminate the row. Core's phrase is the one permanent-reject
condition where the object hash is burned for this cycle.

Regression test: a proxy 429 "rate limit exceeded" body leaves
the row in awaiting_collateral, while Core's exact phrase flips
to failed/submit_rejected on the next tick.

P2 (routes/govProposals.js): wrap the winner re-read in the
unique-constraint recovery path (submissions.findPreparedByData
HexForUser) in try/catch. better-sqlite3 is synchronous and can
throw (SQLITE_BUSY, I/O, corrupt index) from that read; without
the guard the throw escaped into the async Express 4 handler as
an unhandled rejection, which in prod surfaces as a process-
level UnhandledPromiseRejection warning and a client-visible
500 HTML default page instead of the handler's structured
{ error: 'internal' } JSON. Returns 500 JSON on lookup failure,
matching the pattern used for the other DB reads in this route.

Regression test: prime a prepared row so the UNIQUE index will
reject a second insert; make the pre-read miss (drives the
recovery branch) AND throw SQLITE_BUSY on the re-read; assert
the second /prepare returns { error: 'internal' } JSON 500.

Made-with: Cursor

Backend:
- BE P1 (lib/proposalDispatcher.js): mark rows failed with
  duplicate_governance_hash when gObjectSubmit succeeds but the
  repo's UNIQUE index on governance_hash rejects the UPDATE.
  Previously this path only logged and returned, leaving the row
  stuck in awaiting_collateral and retried forever with no
  terminal outcome. Mirrors the existing duplicate-error branch
  and fires onFailed so the user gets a terminal notification.
- BE P2 (lib/proposalValidate.js): parsePaymentAmountToSats now
  handles numeric inputs that JavaScript stringifies to
  scientific notation (e.g. 0.00000001 -> "1e-8"). Route through
  toFixed(8) so the decimal regex accepts them; previously valid
  sats-scale numeric values surfaced as validation_failed.

Frontend:
- FE P2 (src/pages/NewProposal.js): always clear loadError when
  the ?draft= query is removed from the URL. Prior code gated
  the reset on draftId != null, so a failed load (which clears
  draftId in catch) left a stale "Couldn't load draft" banner
  visible on a fresh-new-proposal route.
- FE P2 (src/lib/proposalForm.js): draftBodyFromForm now accepts
  { forUpdate } and, when true, emits explicit empty strings for
  cleared text fields (name/url/paymentAddress) and null for
  cleared epochs. NewProposal passes forUpdate: true to
  updateDraft so user clears actually PATCH through. Previously
  dropping empty keys silently discarded a user's explicit
  delete of e.g. url.
- FE P2 (src/pages/ProposalStatus.js): extend the [id] reset
  effect to also clear deleteError, attachError, and txidInput.
  These are per-submission UI state; leaving them set across
  /governance/proposal/:id navigations showed stale error
  banners on a different row and — worse — prefilled row B's
  attach-collateral textbox with a txid the user pasted into
  row A.
- FE P3 (src/pages/ProposalStatus.js): treat not_found as
  non-retryable in the polling short-circuit. Previously the
  guard excluded only invalid_id and forbidden; a stale/deleted
  submission would otherwise keep re-firing getSubmission every
  60s indefinitely for every mounted tab.

Full regression tests added for each fix; existing test suites
continue to pass (BE 714 / FE 652).

Made-with: Cursor